Choosing the right UI test automation tool

03.Unattended execution. Sometimes I feel this is asking for too much from an UI driven automation tool. But it depends how you want to read it. I read it as some way to schedule my test runs with minimal configuration, plus once kicked-off, it should be self contained to proceed on different situations it might encounter during that run. Now, the later part is actually highly dependent on "how" your automated testing is designed and how well the coder has implemented it. Yeah, it will need some of your intelligence to be smart! Just record-playback is not going to give you a robust system.

04.Ability to capture execution results. A good automation tool should always be able to tell you what just happened. As we are expecting it to be unattended, we want the tool to tell us what happened at the end of each test case (at least) if not log results of each step of execution. So, when the pass is over, the tester can look at the logs to figure out which tests passed and which failed and then take appropirate action. Most of off the shelf tools come with good logging facilities, but again, you will have to be smart to design and code your tests to extend the tool's logging capabilities.

05.Ability to compare "expected" vs "actual" results. So we want logging capabilities in the tool and we also want the tool to be able to compare results to a baseline so that we can make a decision on calling a particular step or even a test case to be passing or failing. This is probably the most important "ask" that you should have when looking out for a tool. You also have to ask deeper question of what all it can compare - text?, numbers?, images? etc. You will find these specific questions looking at the software under test. It is needless to say even if the tool can compare two values, it will be your responsibility to code it right to make it able to take the right decision!